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Introduction. 


Suppose  that  X^, . . . are  independent,  identically  distributed 

d 

random  vectors  with  values  in  1R  and  with  a common  probability  density 

f.  If  V^x)  is  the  volume  of  the  smallest  sphere  centered  at  x and  containing 

at  least  k of  the  random  vectors  X.  , . . . ,X  , then  Loftsgaarden  and  Quesen- 

l n 

berry  (1965) , to  estimate  f(x)  from  X.  , . . . ,X  , let 

i n 

fjj(x)  = k/(nVk(x))  (1) 

where  k = k(n)  is  a sequence  of  positive  integers  satisfying 

(a)  k(n)  t ® 

(2) 

(b)  k(n)/n  -»  0 . 

- 

(The  factor  k-1  was  used  instead  of  k by  Loftsgaarden  and  Quesenberry;  this 
has  no  effect  on  any  of  the  asymptotic  results  stated  here.)  They  showed  that 
fR(x)  is  a consistent  estimate  of  f(x)  at  each  point  where  f is  continuous  and 
positive.  This  result  can  also  easily  be  inferred  from  the  work  of  Fix  and 

1 

Hodges  (1951).  Ford  = 1,  Moore  and  Henrichon  (1969)  showed  that 

sup  |f  (x)  - f(x)  | -♦  0 in  probability 
x 

if  f is  uniformly  continuous  and  positive  on  1R  and  if,  additionally, 

k(n)/log  n -♦  « . (3)  ~ ] 

Wagner  (1973)  showed  that  f^fx)  is  a strongly  consistent  estimate  of  f(x)  at 
each  continuity  point  of  f if,  in  addition  to  (2b) , 


(Notice  that  (4)  is  always  implied  by  (3)  but  (2a)  and  (3)  are  needed  to 
imply  (4).)  The  result  of  this  paper  is  the  following  theorem. 

Theorem . If  f is  uniformly  continuous  on  JR01  and  if  k(n)  satisfies  (2b)  and 
(3)  then 

sup  | fn (x)  - f(x)  | " 0 w.p.l. 
x 

If 

n 

*n(x)  = Zj  K((x-X.)/r(n))/nr(n)d  , 

where  K is  the  uniform  probability  density  for  the  unit  sphere  in  IRd  and  {r(n) } 
is  a sequence  of  positive  numbers,  the  recent  results  of  Moore  and  Yackel 
(1977)  (see  Theorem  3.1)  and  the  above  theorem  Immediately  yield  that 

A 

sup  |f  (x)  - f(x)  I -*  0 w.p.  1 
x n 1 

whenever  f is  uniformly  continuous  on  IRd  and  r(n)  0,  nr(n)d/log  n -♦  ». 

This  fact,  an  improvement  over  the  previously  published  convergence  results 
for  the  kernel  estimate  with  a uniform  kernel  (e.g. , see  Theorem  2.1  of  Moore 
and  Yackel  (1977)),  also  is  a special  case  of  Theorem  4.9  of  Devroye  (1976) 
who  proves  the  same  statement  for  all  kernels  K which  are  bounded  probability 
densities  with  compact  support  and  whose  discontinuity  points  have  a closure 
with  Lebesgue  measure  0. 

Proof. 

To  simplify  notation  we  assume  below  that  multiplications  are  always 
carried  out  before  division.  Let  e > 0 and  choose  6 > 0 such  that 

|f(y)  - f(x)  | < e/2 


- V ..  -r.  * 


whenever  x and  y are  within  a sphere  of  volume  6.  Deferring  measurability 
arguments  for  the  moment, 


P {sup  |fn(x)  - f(x)  I > e}  = 


P{|j[Vk(x)  < k/n(f(x)  + e)]}  + 
x 

P{  U [V.(x)  > k/n(f(x)  - e)]}  . 
x:f(x)>  e 


The  event  u[Vk(x)  < k/n(f(x)  + e)]  implies  that,  for  some  x,  there  must  be 
x 

a sphere  centered  at  x with  volume  less  than  k/n(f(x)  +•  e)  and  containing  k 

of  the  random  vectors  X^, . . . ,Xr.  If  k/ne  < 6 then  the  probability  measure 

k(f(x)  4"  6 /2) 

of  such  a sphere  must  be  less  than  ' /z/ . “ so  that,  for  one  of  these 

n(f(x)  + e) 

spheres  S, 


u (SI  - u(S)  > — - k.(f(x)  + e/.2) 

ii(S)  > n n(f(x)  + e) 


n 


k e 


ke 


2n(f(x)  + e)  2n(F  + e) 


where  F is  the  maximum  of  f on  Rd , p,  is  the  measure  on  the  Borel  subsets  of 

corresponding  to  f and  is  the  empirical  measure  on  the  Borel  subsets 

of  Rd  for  X. , . . . ,X  . Thus , for  k/ne  <6, 
i n 


P { U [Vk(x)  < k/n(f(x)  + e)]  } s 
x 

P { sup  I Li  (S)  - p,(S)  I >ke/2n(F+  e)}  (5) 

SeC 


where  G is  the  class  of  all  spheres  in  lRd  whose  volume  is  less  than  4k/ne 


Next,  with  4k/ne  < 6, 


U [V.  (x)  > k/n(f(x)-e  )]c 
x:f(x)>  e 

U [V  (x)  >k/n(f(x)  - (3e/4))] 
x:f(x)>  e 


which  implies  that,  for  some  x with  f(x)  > e,  there  is  a sphere  S centered  at 
x,  with  volume  s 4k/ne  , and 

»»(S)  *k(f(x)  - e/2)/n(f(x)  - (3/4)  e) , 

Hn(S)  s k/n , and 

H(S)  - un(S)  s k«/4n(f(x)  - (3/4)  e)  . 

Thus 

P{  U [V .(x)  >k/n(f(x)  - •)])  £ 

x:f(x)>  e {6) 

P{  sup  |n(S)  - u (S)  | s ke/4nF } , 

Sc  G n 

n 

so  that 


P{sup  | f (x)  - f(x)  | & e}  *2P{  sup  j,*  (S)  - ^(S)  | :>  ke/4n(F+e) } . 

x SeG  n 

n 


The  proof  will  be  completed  if  we  show  that  for  each  e > 0 


2>{sup  U„(S)  - ji(S)  | a ke/4n(F+e)}  < ® . (7) 

n SeG  n 
n 


To  prove  (7)  we  employ  a variation  of  the  argument  used  by  Vapnik 
and  Chervonenkis  (1971).  In  this  variation  use  will  be  made  of  the  following 
result.  If  Yj  , . . . ,Yn  represent  independent  drawings  without  replacement 


from  a population  of  k 0's  and  l's  then,  for  e > 0 and  kin, 


(8) 


where  p,,  the  {number  of  l's}A/  is  assumed  to  be  £ £.  Additionally  (8) 

holds  when  Y^ , . . . , Y^  are  Bernoulli  random  variables  with  parameter  w s ^ . 

(Use  the  two-sided  version  of  Theorem  3 of  Hoeffding  (1963)  along  with 

p,  s f and  log  (1  + (c/p,))  2 2c/(2p  + e).  See  also  section  6 of  this  paper.) 

2 

Now,  if  supp(G)  £ M and  n 2 8M/6  , an  easy  modification  of  Lemma 
G 

1 of  Vapnik  and  Chervonenkis  (1971)  yields 


Pfsup  |pn(A)  - ^ (A)  | 2 6]  £ 
G 


2P[sup  |pn(A)  - u^(A)  | a 6/2] 
G 


(9) 


where  p,j^(A)  is  the  empirical  measure  for  A with 
class  of  Borel  sets  in  TRd  for  which 


,X_  and  G is  any 
zn 


sup  |un(A)  - p,(A)  | and 
G 

sup  | pn(A)  - p^(A)  | 

G 


are  random  variables . Putting  G = G we  see  that  M can  be  taken  to  be 

n 

4kF/ne.  Since,  for  a > 0, 


P[sup  | p.  (A)  - p,'  (A)  | a 6/2]  £ 


n 


P[sup  Un(A)  - p^(A)  | 2 6/2  ; sup  p2n(A)  £ erM] 

G C 

n n 

+ P[sup  p2 n(A)  > aM] 

Si 


(10) 


we  see,  using  (3)  and  putting  6 = ke/4n(F  + e),  that  (7)  follows  whenever 
both  terms  of  the  right-hand  side  of  (10)  are  summable  for  some  a > 0. 
Looking  at  the  first  term,  we  note  that  it  equals 


^ 2nd  (2n)  !^X[sup  |^n(A)  - y,^(A)  | s e/Zl^sup  u-2n(A)  s «M]dC? 


n 


n 


where  I is  the  indicator  of  the  set  E c IR  and  Q is  the  probability  measure 

£i 

2 nd 

on  1R  for  X, , . . . ,X„  and  where  the  inner  summation  is  taken  over  all 
1 in 

(2n) ! permutations  of  x^ , . . . ,x2n.  But  t^lis  last  integral  equals 

• .And  (2n) ! ^[sup  ti2n(A)sarM]  S£P  *[  |^n(A)  - M-^(A)  I * 6/2]dQ 
K n n 


n 


dQ 


^2nd  (2n)  1 ^^sup  ^2n(A)i»M]  ^J?I[|un(A)  - p,^(A)  | ^ 6/2] 
s .^2nd  *[sup  n2n(A)s»M]  | (2n) ! ^ *[  i y>n(A)  - p,2n(A)  I * 6/4^ 


dQ 


n 


where  G'  = G '(x. , . . . ,x_  ) is  any  finite  subclass  of  G which  yields  the  same 
l zn  n 

class  of  intersections  with  fx. , . . . ,x„  } and  where  the  inner  summation  is 

l zn 

again  taken  over  the  (2n) ! permutations  of  x^  , . . . ,x2n<  quantity  within 
{•  } is  bounded  above,  using  (8) , by 

2e-n62/(32^n(A)  + 46) 

whenever  M-2n(A)  s:  Since  M = 4kF/ne  we  see,  from  (3) , that  for  all  n 


sufficiently  large  the  last  integral  is  upper-bounded  by 


vnde 


-n  6 / (3  2 o M+4  6)/ 


dQ  . 


Choosing  G’  to  be  a smallest  pc  *ible  subclass,  we  have  (Vapnik  and 


Chervonenkis  (1971),  Cover  (1965))  that 


(£>) 

\AeG7 


s 1 + (2n)  and , using 


(3)  again,  that  the  first  term  of  (10)  is  summable  for  all  a > 0. 

Looking  at  the  second  term  of  (10) , let  r be  the  radius  of  a sphere  in 

JRd  whose  volume  is  4k/ne.  If  some  sphere  of  radius  r contains  £ of  the 

points  X , . . . ,X„  then  there  must  be  at  least  one  sphere  of  radius  2r , 
x l n 

centered  at  one  of  the  points  X^ , . . . /^n'  which  contains  at  least  £ points. 
Thus 


P[sup  ^^(A)  > S ^nP^2n^X  > 

Gn  1 ' . 

where  Sx(t)  denotes  the  sphere  of  radius  t centered  at  x.  But 

p[n2n(sx  (2r))  >aM]  * 


max  P[ii.  .(S  (2r))  > («2nM-l)/(2n-l)] 
2n-l  X 


imax  PU0  AS  (2r))  > [(«2nM-l)/(2n-l)]  -2  4kF/ne]  . 

2n-i  x 

xelRa 

At  this  point  it  is  not  difficult,  using  (3)  and  (8) , to  show  that  the  second 
term  of  (9)  is  summable  as  long  as  a > 2d. 

Finally,  to  complete  the  proof,  it  is  easy  to  see  that  all  of  the  un- 
countable unions  over  x are  indeed  events  and  that  the  various  supremums 

over  G are  indeed  random  variables . 
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